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INDIVIDUAL  FORECASTING  ABILITY 

1.   Introduction 

Are  some  analysts  "superstars"?   Financial  press  coverage  of  analysts' 
performances  suggests  that  there  are  superior  financial  analysts.   One  example 
of  this  coverage  is  the  Institutional  Investor  "All  American  Research  Team," 
based  on  surveys  of  money  managers,  who  nominate  and  rank  analysts.   In  this 
paper,  I  explore  the  question  of  individual  superiority  by  examining  one  aspect 
of  analysts'  services:   earnings  forecasting.    Forecasts  of  future  earnings 
are  of  interest  to  investors,  as  inputs  to  investment  decisions,  and  also  to 
academics,  as  data  on  earnings  expectations. 

Academic  use  of  analysts'  earnings  forecasts  as  proxies  for  expected 
earnings  m  capital  markets  research  is  now  widespread.    Both  availability  of 
large  databases  of  forecasts  and  demonstrations  of  analysts'  accuracy  at 
prediction  have  aided  this  development.   The  results  of  this  paper  are  useful 
to  academics  interested  in  using  analyst  forecasts  as  earnings  expectations. 
If  individual  analysts'  forecasting  ability  differs  in  a  systematic  way  that 
remains  reasonably  stable  over  time,  this  information  can  be  used  to  construct 
better  aggregate  forecasts,  by  forming  weighted  combinations.   I  find  no 


The  Institutional  Investor  list  is  based  on  several  criteria,  inclufUng 
earnings  forecasting,  picking  stocks,  and  the  quality  of  written  reports. 
By  focusing  on  earnings  forecasting,  I  do  not  mean  to  suggest  that  it  is  the 
only  or  the  most  important  of  analysts'  activities. 

2 
Some  examples  are:   Patell  and  Wolfson  (1984),  Ricks  and  Hughes  (1985), 

Bamber  (1986),  Hoskin,  Hughes  and  Ricks  (1986)  and  Pound  (1987). 

Examples  of  such  databases  are:   the  I/B/E/S  Summary  data,  the  Standard 
and  Poors'  Earnings  Forecaster,  and  Zacks  Investment  Advisory  Services' 
database. 

^Brown  and  Rozeff  (1978),  Collins  and  Hopwood  (1980),  Fried  and  Givoly 
(1982) ,  O'Brien  (1987) . 
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evidence  that  this  effort  is  likely  to  be  fruitful.   Rather,  my  results  suggest 
that  individuals  are  relatively  homogeneous  in  terms  of  their  forecast 
accuracy. 

The  criterion  I  apply  to  evaluate  forecasting  ability  is  average  accuracy  on 
the  portfolio  of  predictions  made  by  an  individual,  through  time.   It  is 
reasonable  to  presume  that  some  firms'  earnings  are  harder  to  forecast  than 
others'.   Since  analysts  may,  to  some  extent,  choose  to  predict  earnings  for 
some  firms  and  not  others,  this  endogenous  selection  is  important  to  the 
specification  of  statistical  tests.   The  results  reported  here  remove  the 
effects  differential  predictability  across  industries  in  comparing  analysts. 

Evaluation  of  average  portfolio  accuracy  through  time  is  important  in 
earnings  forecast  data.   Each  year,  there  is  a  relatively  large  component  of 
unanticipated  information  which  is  common  across  firms,  probably  because  of 
unanticipated  economy-wide  or  industry-wide  events.   This  common  information  is 
evident  in  the  large,  statistically  significant  year-by-year  differences  in 
(cross-sectional)  average  forecast  errors.   This  general  feature  of  earnings 
forecast  data  makes  it  difficult  to  distinguish  between  a  "lucky  guess"  on  a 
macro  event  and  true  forecasting  ability  at  the  micro  level,  when  only  short 
time-series  are  available.    Current  databases  are  rich  in  cross-sectional 
observations,  but  the  number  of  years  over  which  forecasts  are  available  is 
still  relatively  small.   This  issue,  and  my  treatment  of  it,  is  discussed 
further  in  section  3. 

The  paper  is  organized  as  follows.   Because  the  database  used  is  not  widely 
available,  section  2  describes  the  data  in  some  detail.   Section  3  concerns 


Forecasts  of  macroeconomic  variables  are  of  interest  in  their  own  right. 
They  have  been  studied,  for  example,  by  Zarnowitz  (1984),  and  others.   However, 
my  focus  is  on  forecasting  ability  at  the  portfolio  or  individual  firm  level, 
which  is  ostensibly  a  component  of  analysts'  services  to  their  clients. 
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differences  in  portfolios  of  firnns  forecast  by  different  individuals,  and  the 
implications  for  sample  selection.   In  section  4  I  describe  the  statistical 
model  used  to  examine  the  relative  accuracy  of  individual  analysts'  forecasts, 
and  present  the  evidence  from  this  sample.   Section  5  is  a  summary  with  some 
concluding  remarks. 


2.   The  Analyst  Forecast  Data 

The  data  are  individual  analyst  forecasts  made  between  1975  and  1981, 
compiled  by  Lynch,  Jones  and  Ryan,  a  New  York  brokerage  house.   Lynch,  Jones, 
and  Ryan  use  the  individual  forecasts  to  produce  the  I/B/E/S  Summary  database, 
which  is  sold  to  clients,  who  are  primarily  institutional  investors.   The 
analysts  included  in  tlip  database  are  employed  by  a  variety  of  financial 
service  institutions,  such  as  brokerage  houses  and  investment  banks.   Analysts 
and  their  employers  are  denoted  by  code  numbers  in  the  database,  and  I  do  not 
have  access  to  information  on  their  identities. 

The  database  is  updated  once  per  month  with  new  forecasts.   The  monthly 
lists  used  in  this  study  span  July  1975  through  September  1982.   Data  items 
include  the  identity  of  the  firm  whose  earnings  per  share  (EPS)  are  forecast, 
the  identities  (code  numbers)  of  the  individual  making  the  forecast  and  his 
employer,  and  the  date  the  forecast  was  made.   Although  most  financial  set  vice 
institutions  (hereaftei-,  FSIs)  employ  many  analysts,  at  most  one  analyst's 
forecast  is  available  from  each  FSI  for  a  given  firm's  earnings  at  a  given 
time.   Forecasts  are  collected  by  Lynch,  Jones  and  Ryan  for  the  current  fiscal 
year,  and  for  the  following  year  when  available.   I  use  only  forecasts  for  the 
current  fiscal  year. 
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The  number  of  firms  whose  earnings  are  forecast  in  the  I/B/E/S  database  has 
grown  considerably  over  time,  from  approximately  1000  in  1975  to  more  than  2500 
by  1982.   My  initial  subset  of  the  database  comprises  the  508  firms  with 
December  yearends  which  have  at  least  one  forecast  available  in  each  year,  1975 
through  1981.   This  requirement  will  impart  a  "survivorship"  bias  to  the 
sample,  since  firms  that  started  up  and  firms  that  ceased  to  exist  between  1976 
and  1981  are  excluded.   Also,  since  I/B/E/S  has  grown  through  time,  the  firms 
with  forecasts  available  in  1975  may  have  been  those  most  closely-followed  by 
financial  analysts,  and  therefore  may  be  larger  and/or  more  stable.   To  the 
extent  that  this  was  true  of  the  database  in  1975,  it  will  be  carried  over  into 
my  sample. 

A  firm  is  eliminated  from  the  sample  if  annual  earnings  are  not  available  on 
COMPUSTAT  in  1982.   This  requirement  reduces  the  sample  to  497  firms,  and 
probably  also  increases  the  bias  toward  larger  and/or  more  stable  firms. 

The  identification  of  forecasts  with  individual  analysts  is  a  valuable 
feature  of  this  database.   This  feature  allows  me  to  examine  forecasting 
ability  at  a  more  micro  level  than  has  been  done  in  previous  studies.   However, 
it  is  worth  noting  two  important  features  of  the  data,  and  my  attempts  to 
address  them  in  designing  the  empirical  study.   The  first  feature  is  the  non- 
uniqueness  of  analyst  identifier  codes.   The  second  feature  is  the  "publication 
lag"  in  the  data.   These  are  decribed  in  turn  below. 

To  be  most  useful  for  this  type  of  study,  the  identifying  codes  for 
individual  analysts  should  uniquely  identify  a  person  for  the  entire  time 
period  he  produces  forecasts,  and  regardless  of  the  FSI(s)  employing  him. 
While  this  is  generally  true  of  the  analyst  identifiers  in  the  database,  I 
found  instances  of  non-unique  codes,-  that  is,  the  same  code  was  given  to  more 
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than  one  individual  in  the  database  at  one  point  in  time.   However,  the 
combination  of  the  FST  and  analyst  codes  together  gave  unique  identification. 

To  avoid  attributing  forecasts  made  by  two  different  people  to  a  single 
person,  I  use  the  combination  of  FSI  and  analyst  codes  to  identify  individuals 
in  this  study.   The  drawback  to  this  approach  is  that  it  does  not  allow 
individuals  who  change  jobs,  but  remain  in  the  database  employed  by  a  different 
FSI,  to  be  tracked  through  time.   Therefore,  analysts  will  appear  in  my  sample 
to  predict  for  smaller  numbers  of  years,  on  average,  than  is  the  case  in  the 
Real  World.   Because  I/B/E/S  is  a  secondary  source,  and  I  do  not  have  access  to 
the  identities  of  FSIs  or  individuals,  this  limitation  seems  unavoidable. 

The  data  are  constructed  from  self-reports  of  forecasts  by  analysts  to 
Lynch,  Jones  &  Ryan,  along  with  regular  telephone  surveys  of  analysts  who  fail 
to  report.   Analysts  assign  forecast  dates  to  the  forecasts,  and  these  forecast 
dates  are  reported  in  the  database.   There  is  a  "publication  lag"  between  a 
forecast's  date  and  its  first  appearance  in  the  I/B/E/S  database,  which  arises 
for  the  following  reasons.   First,  because  the  data  from  which  my  sample  are 
drawn  are  based  on  monthly  updates  of  forecasts,  and  it  is  implausible  that  all 
analysts  issue  forecasts  immediately  prior  to  the  updates,  a  publication  lag  of 
less  than  one  month,  on  average,  is  inevitable.   Second,  an  additional  lag  is 
created  by  (possibly  unavoidable)  imperfections  in  the  self-reporting  and 
survey  process.   The  result  is  that  some  updates  made  within  a  given  month  do 
not  appear  in  I/B/E/S  until  a  later  month.   On  average,  in  the  508-firm  sample, 
over  the  seven  years  from  197S  through  1981,  the  publication  lag  is  34  trading 
days,  or  about  one-and-a-half  calendar  months,  with  a  standard  deviation  of 
44.5  trading  days. 

In  selecting  the  sample  for  this  study,   I  eliminate  publication  lags,  by 


The  sample  selection  is  described  in  section  3, 
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ignoring  the  date  of  the  forecast's  first  appearance  in  I/B/E/S,  and  using  the 
analyst's  forecast  date.   After  eliminating  publication  lags,  there  is  still 
considerable  variation  m  the  ages  of  forecasts   in  the  sample.   These 
differences  exist  primarily  because  analysts  differ  in  how  frequently  they 
update  their  forecasts.   The  more  frequently  an  analyst  updates  forecasts,  the 
smaller  will  be  the  average  age  of  his  forecasts  included  in  the  sample. 
However,  unlike  the  publication  lag,  which  may  or  may  not  be  under  the  control 
of  the  analyst,  and  which  is  unlikely  to  be  related  to  forecasting  ability,  the 
frequency  with  which  t-he  analyst  updates  forecasts  is  clearly  under  his 
control,  and  is  more  likely  to  be  related  to  forecasting  ability.   Therefore, 
variation  in  the  ages  of  forecasts  included  in  the  sample  reflects  a 
potentially  important  feature  of  the  population  from  which  the  sample  is  drawn. 
I  return  to  this  issue  in  the  following  section. 


3.   Sample  Selection 

The  sample  is  selected  at  a  fixed  horizon  of  120  trading  days,  or  slightly 
less  than  six  calendar  months,  prior  to  the  announcement  of  annual  earnings. 
That  is,  for  each  firm  and  year,  the  announcement  date  of  annual  earnings  is 
obtained  from  COMPUSTAT  or  from  the  Wall  Street  Journal.   Next,  the  horizon 
date,  120  tradings  days  prior  to  the  announcement,  is  determined  using  tl°  CRSP 
trading  day  calendar.   From  the  point  of  view  of  the  horizon  date,  the  m^st 
recent  forecast  from  each  analyst  forecasting  EPS  for  that  firm  and  year  is 
selected  for  the  sample. 


7 
A  forecast's  age  is  defined  as  the  length  of  time  between  the  analyst's 

forecast  date  and  the  (exogenously  chosen)  date  at  which  the  sample  is 

constructed.   That  is:  from  the  point  of  view  of  the  sample  selection  date,  how 

old  is  a  given  forecast?   See  O'Brien  (1987)  for  an  analysis  of  forecast  ages 

in  a  different  sample  from  the  same  database. 
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The  motivation  for  selecting  a  fixed  horizon  is  to  make  the  date  at  which 
the  sample  is  constructed  independent  of  analysts'  decisions,  and  to  make  the 
horizons  roughly  comparable  across  firms  and  years.   This  sample  of  forecasts 
is  intended  to  represent  the  population  available  at  a  fixed,  exogenously 
determined,  point  in  time. 

The  analysts'  forecast  data  form  a  panel,  or  time-series  of  cross-sections. 
Because  analysts  can,  to  some  extent,  choose  the  firms  they  forecast  and 
because  of  limitations  on  individuals'  time  and  energy,  forecast  data  are 
extremely  unbalanced.   That  is,  there  is  not  a  forecast  from  each  analyst  for 

o 

each  firm  in  each  year.    The  fact  that  EPS  forecast  data  are  inherently 
unbalanced  has  important  implications  for  the  sample  design  and  statistical 
inferences. 

A  common  sample  design  involves  choosing  a  set  of  firms  and  a  set  of 
analysts  for  which  complete  data  (in  this  case,  a  forecast)  can  be  obtained  for 
each  time  period.   Tlii'^  sample  design  results  in  balanced  data.   An  advantage 
to  analyzing  balanced  data,  in  evaluating  predictive  ability  of  individual 
forecasters,  is  that  all  forecasters  are  evaluated  on  the  same  benchmark,  since 
by  sample  construction  each  forecaster  predicts  the  same  firms'  earnings  in  the 
same  years. 

On  the  other  hand,  this  sample  selection  procedure  can  obscure  important 
variation  in  the  sample.   For  example,  in  the  database  from  which  this  sample 
is  drawn,  to  obtain  balanced  data  on  a  five-year  time-series,  it  is  necessary 
to  limit  the  portfolio  to  at  most  fifteen  firms  at  one  time.   More  importantly, 
comparisons  are  limited  to  at  most  six  analysts  at  one  time.   Clearly,  this 


Q 

This  feature  is  not  unique  to  I/B/E/S.   For  example,  a  perusal  of  the 
Standard  and  Poors'  Earnings  Forecaster  will  verify  that,  even  at  the  level  of 
FSIs,  the  data  are  unbalanced. 
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limits  the  researcher's  ability  to  generalize  results  to  the  larger  population 
of  analysts  and  firms. 

The  tradeoff,  in  selecting  a  sample  of  analysts'  forecasts,  is  between 
preserving  enough  data  to  make  comparisons  meaningful  and  maintaining  enough 
variation  in  the  sample  to  allow  generalization.   At  one  extreme,  if  each 
analyst  predicts  EPS  for  only  one  firm  in  one  year,  then  comparisons  across 
analysts  are  not  statistically  meaningful.   At  the  other  extreme,  if  the  sample 
is  selected  to  achieve  fully  balanced  data  as  described  in  the  previous 
paragraph,  statistic?*!  comparisons  can  be  made,  but  generalization  of  the 
inferences  to  other  analysts  and  firms  would  likely  be  unreliable. 

An  alternate  way  to  treat  unbalanced  data  is  to  adjust  for  the  effects  of 
the  lack  of  balance  in  the  statistical  design.   This  is  the  approach  I  use. 
The  statistical  techniques  are  described  in  section  4.   Sample  selection 
criteria  (4)  and  (5),  described  below  and  in  Table  1,  are  attempts  to  address 
the  tradeoff  discussed  above:   retaining  as  much  variation  as  possible,  while 
allowing  statistical  comparison. 

The  first  two  sets  of  criteria  listed  in  Table  1  are  those  described  in  the 
previous  section,  resulting  in  497  firms.   Criterion  (3)  describes  the  set  of 
forecasts  available  in  the  I/B/E/S  database  at  the  120-trading-day  horizon  for 
the  sample  firms.   There  are  38611  forecasts  of  457  firms'  EPS  from  3041 
analysts  at  this  stage.   The  effects  other  sample  selection  criteria  on  the 
number  of  forecasts,  firms  and  analysts  in  the  sample  are  displayed  in  Table  1 
and  discussed  below. 

Criterion  (4),  that  analysts  in  the  sample  have  at  least  30  forecasts,  for 
any  firms  in  any  years,  eliminates  nearly  90%  of  all  analysts  in  the  database. 
However,  note  that  the  remaining  approximately  10%  of  analysts  produce  nearly 
half  of  the  forecasts.   Criterion  (4)  is  clearly  affected  by  the  conservative 
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method  of  uniquely  identifying  analysts,  discussed  in  the  previous  section. 
The  method  limits  the  possibility  of  attributing  forecasts  made  by  two 
different  individuals  to  a  single  analyst,  but  results  in  understatement  of  the 
number  of  forecasts  available  from  individuals  who  change  jobs.   The  reason  for 
criterion  (4)  is  to  ensure  sufficient  information  is  available  to  generate 
statistically  viable  comparisons. 

The  fifth  criterion,  that  each  analyst  in  the  sample  have  forecasts  in  at 
least  four  of  the  seven  years  for  which  data  exist,  again  addresses  the  issue 
of  statistical  validity.   As  was  discussed  previously,  the  shorter  the  time- 
series  on  a  given  analyst,  the  harder  it  is  to  distinguish  true  forecasting 
ability,  because  of  common  information  across  firms  within  years. 

Criterion  (6)  attempts  to  address  a  possible  flaw  in  the  database.   As 
mentioned  above,  at  any  point  in  time,  there  is  variation  in  the  ages  of 
forecasts  available.   This  variation  may  occur  in  the  population  for  several 
reasons.   First,  if  some  analysts  update  their  forecasts  only  in  response  to 
new  information,  old  forecasts  may  remain  because  (1)  for  some  firms,  there  has 
been  no  important  unanticipated  information,  or  (2)  the  analyst  is  a  superior 
forecaster  and  foresaw  events  which  others  did  not,  hence  no  updating  was 
necessary  for  that  individual .   Second,  analysts  who  are  less  diligent  or  less 
interested  than  their  peers  may  update  less  frequently.   Both  these  sources  of 
variation  in  age  of  forecasts  are  features  of  the  population  that  may  be 
related  to  forecasting  ability,  and  therefore  are  desirable  to  retain  in  the 
sample . 

A  third  possible  source  of  variation  in  forecast  ages  is  attributable  to  the 
data  collection  process,  and  therefore  is  not  desirable  to  retain  in  the 
sample.   Forecasts,  once  submitted,  are  retained  in  the  database  until  they  are 
(1)  updated  by  the  analyst,  (2)  replaced  by  another  forecast  from  a  different 
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analyst  at  the  same  FSI,  (3)  withdrawn  by  the  analyst,   or  (4)  made  obsolete  by 
the  announcement  of  actual  EPS  for  that  firm  and  year.   If,  however,  the 
analyst  or  FSI  stops  submitting  forecasts  to  I/B/E/S,  the  forecast  might 
inadvertently  remain  in  the  database. 

The  effects  on  the  sample  of  eliminating  forecasts  on  the  basis  of  forecast 
age,  given  all  the  other  sample  selection  criteria,  are  displayed  in  Table  2. 
The  goal,  in  applying  an  age  filter,  is  to  eliminate  the  very  oldest  forecasts, 
which  are  most  likely  to  be  artifacts  of  the  data  collection  process,  without 
eliminating  the  desirable  variation  in  ages  in  the  sample.   An  interesting 
feature  to  note  in  Table  2  is  that  the  variation  in  ages  of  forecasts  in  the 
sample  is  primarily  attributable  to  differences  across  analysts,  not  across 
firms.   This  is  evident  from  the  fact  that  reductions  in  the  number  of 
forecasts  from  successively  stronger  age  filters  are  roughly  proportional  to 
reductions  in  the  number  of  analysts,  and  not  to  reductions  in  the  number  of 
firms. 

Criterion  (6)  in  Table  .1  is  a  filter  on  forecast  age.   Forecasts  which  are 
more  than  200  trading  days  (about  9  calendar  months)  old  on  the  horizon  date 
are  eliminated.   Twenty-two  analysts  are  eliminated  from  the  sample  by  this 
criterion. 

In  the  sample  that  results  from  these  criteria,  there  are  10,586  forecasts 
from  191  analysts  for  42R  firms.   The  major  industries,  defined  by  2-digit  SIC 
codes  from  COMPUSTAT,  represented  in  the  sample  are  displayed  in  Table  3.   The 
data  are  not  concentrated  by  industry.   Not  surprisingly,  some  industries 
appear  to  be  more  closely-followed  by  analysts  than  others.   For  example,  the 


g 
This  can  occur,  for  example,  with  an  important  new  event,  such  as  a 

tender  offer  or  a  strike,  which  may  alter  estimates  substantially.   The  analyst 

will  temporarily  withdraw  his  forecast  to  study  the  implications  of  the  new 

event,  and  issue  a  new  one  later. 
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Petroleum  refining  industry  (SIC  29)  accounts  for  only  4.7%  of  the  firms  in  the 
sample,  but  represents  10.9%  of  the  forecasts.   On  the  other  hand,  Electrical 
and  electronic  (SIC  36)  with  5.4%  of  the  firms,  has  only  2.9%  of  the  sample 
forecasts . 

Eight  of  the  10,586  forecasts,  identified  as  probable  data  errors,  are 
eliminated  from  the  sample.   These  eight  are  distinguished  from  other 
"outliers,"  or  large  forecast  errors,  by  comparison  with  other  forecasts  for 
the  same  firm  and  year,  and  with  other  forecasts  by  the  same  analyst  for  other 
firms  and  other  years.   For  each  of  these  eight  observations,  the  forecast 
error  was:   more  that  $10.00  per  share  in  absolute  value,-  more  than  150%  of  the 
prior  year's  actual  EPS;  different  from  other  analysts'  forecast  errors  for  the 
same  firm  and  year  by  an  order  of  magnitude  or  more;  and  different  from  the 
analyst's  average  accuracy  in  other  forecasts  by  an  order  of  magnitude  or  more. 
Such  errors  could  occur,  for  example,  if  a  forecast  of  $2.40  per  share  were 
inadvertently  entered  into  the  database  as  $24.00  per  share. 

In  the  next  section,  the  statistical  design  to  compare  portfolio  performance 
across  analysts  through  time  in  these  data  is  described. 


4.   Statistical  Tests 

All  statistical  tests  in  this  paper  are  based  on  forecast  errors,  comoited 
as  the  difference  between  primary  EPS  for  a  given  firm  and  year  as  reported  in 
COMPUSTAT,  Aj^,  and  the  analyst's  forecast  of  those  EPS,  F^-^: 

e. .^  =  A.^  -  F- .^   ■  (1) 

ijt    jt    i]t  ^  ' 

In  (1),  the  subscript  i  indexes  analysts,  j  indexes  firms  and  t  indexes  years. 
When  a  forecast  is  for  fully-diluted  EPS,  instead  of  primary,  the  forecast 
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number  is  converted  from  fully-diluted  to  primary  using  the  ratio  of  fully- 
diluted  to  primary  EPS  for  that  firm  and  year  from  COMPUSTAT.   When  a  stock 
split  or  stock  dividend  is  announced  between  the  forecast  date  and  the 
announcement  date  of  EPS,  the  forecast  is  adjusted  for  the  capitalization 
change . 

Forecast  errors  constructed  as  in  (1)  are  expected  to  have  large  components 
of  common  information  cross-sectionally  within  years,  as  a  result  of 
unanticipated  macroecomir  events  which  affect  many  firms  in  the  same  way.   This 
is  especially  true  over  long  time  horizons,  like  the  one  used  here  of 
approximately  6  months.    Examples  of  macroeconomic  events  which  can  affect 
many  firms  in  similar  ways  are  unanticipated  changes  in  interest  rates, 
inflation,  or  oil  supply  conditions. 

The  year-specific  components  of  forecast  errors  in  this  sample  are 
constructed  using  the  following  model: 

e..   =  ii  +   E..     .  (2) 

ijt    ^t     ijt  ^    ' 

The  V'Y'    °^   annual  average  forecast  errors,  in  equation  (2)  are  estimated  using 
OLS  on  a  set  of  dummy  variables  for  the  seven  years  in  the  sample.   The  results 
of  this  estimation  are  reported  in  Table  4.   It  is  evident  from  Table  4  that 
year-to-year  differences  in  average  forecast  errors  exist,  and  are 
statistically  significant.   The  explicit  statistical  test  for  homogeneity  of 
average  forecast  errors  across  years  is  the  reported  F-statistic  of  35.99, 
which  rejects  the  hypothesis  of  homogeneity.   This  test  assumes  that  deviations 
of  forecast  errors  from  their  annual  averages,  the  E^-jt'  ^re  independent.   If 


See  O'Brien  (1987)  for  evidence  that  this  common  information  component 
increases  as  the  forecasting  horizon  lengthens. 
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adjusted  for  the  possibility  of  heterogeneity  among  analysts  and  among  2-digit 
SIC  industries,    the  test  statistic  becomes  33.44,  which  also  rejects  the 
null. 

Statistical  tests  to  compare  analysts'  forecasting  abilities  are  conducted 
using  the  £;-;4-,  the  deviations  of  forecast  errors  from  the  annual  averages. 
Essentially,  this  purges  forecast  errors  of  aggregate  unanticipated  year- 
specific  information.   This  adjustment  is  important  for  two  reasons,  one  of 
which  is  related  to  the  unbalanced  nature  of  the  data.   First,  given  the 
evidence  in  Table  4  that  years  are  not  homogeneous,  if  the  year-specific 
information  is  ignored,  comparisons  will  depend  upon  the  years  involved.   For 
example,  imagine  one  analyst  with  forecasts  only  in  1975,  1978,  1979,  and  1980, 
the  four  years  with  the  smallest  average  forecast  errors.   Imagine  a  second 
analyst  with  forecasts  only  in  1976,  1977,  1979,  and  1981,  the  four  years  with 
the  largest  average  forecast  errors.   Assume  each  has  (for  simplicity)  one 
forecast  in  each  year,  and  each  is  of  average  forecasting  ability,  as  defined 
by  the  numbers  in  Table  4.   If  forecast  errors  were  assumed  homogeneous  across 

years,  then  the  first  analyst's  average  squared  forecast  error  would  be  .0072, 

1 2 
while  the  second  analyst's  would  be  .0645.    The  first  analyst's  apparent 

greater  accuracy  is  entirely  attributable  to  the  years  involved,  since  each  is 

precisely  average  in  forecasting  ability.   Clearly,  this  feature  exists  only 

because  we  seek  to  make  inferences  from  unbalanced  data.   If  both  analysts  had 


The  test  was  also  done  allowing  for  heterogeneity  at  the  firm,  rather 
than  industry,  level.   The  resulting  F  statistic  was  between  the  two  values 
reported  here. 

12 

Root-mean-squared  errors  may  be  easier  to  interpret  here,  since  they  are 

expressed  in  dollars  per  share,  as  are  EPS  numbers.   The  root-mean-square 
errors  for  this  example  are  $0.08  and  $0.25  per  share  for  the  first  and  second 
analysts,  respectively. 
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forecasts  of  average  accuracy  in  each  of  the  seven  years  they  would  be 
indistinguishable . 

The  second  effect  of  ignoring  annual  average  forecast  errors  in  statistical 
comparisons  is  that  standard  errors  will  be  mis-stated,  because  of  cross- 
sectional  correlation  within  years.   This  is  evident  from  inspection  of 
equation  (2).   An  annual  average  p*.  is  a  component  of  forecast  errors  e--^^, 
common  to  all  analysts  and  firms  in  year  t.   The  t^-^   are  purged  of  this  common 
information. 

The  search  for  superior  ability  to  predict  EPS  is  based  on  £■  •^^,  the 
forecast  errors  purged  of  year-specific  common  unanticipated  information,  or 


deviations  about  the  annual  average.   The  criterion  I  use  to  examine  accuracy 
is  average  square 
effects  model: 


is  average  squared  deviation,  or  the  average  of  (e^.^.)  ,  estimated  in  a  fixed 


^^ijt^'  =   ^1  '    ^2k  '   ^2t  '   ^ijt  <3> 


In  equation  (3),  i,  j  and  t    are  defined  as  before,  as  indexes  of  analysts, 
firms  and  years  respectively.   The  index  k  denotes  the  2-digit  level  industry 
to  which  firm  j  belongs.   The  estimated  effects,  y^^ ,  V2   and  VI3,  are  interpreted 
as  follows,   p, ■  is  the  average  squared  error  accuracy  of  analyst  i, 
conditional  on  the  industries  and  years  in  the  sample  that  are  associated  with 
analyst  i.   That  is,  \i-^^   measures  the  accuracy  of  analyst  i  on  the  portfolio 
for  which  he  forecasts  earnings,  through  time,  v^u    is  the  average  squared 
error  accuracy  for  industry  k,  conditional  on  the  analysts  and  years  in  the 


13 

Searle  (1971)  contains  a  thorough  discussion  fixed  effects  models  and 

their  applications  for  unbalanced  data.   See  Mundlak  (1978)  for  a  general 

exposition  in  economic  contexts. 
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sample.     \i^^   is  the  average  squared  error  accuracy  in  year  t,  conditional  on 
the  analysts  and  firms  in  the  sample. 

The  reason  for  including  firm  and  year  effects  in  equation  (3)  is  to  control 
for  inherent  differences  in  accuracy,  reflecting  differences  in  predictability, 
among  industries  and  years.   Differences  in  predictability  among  industries 
arise  if  there  is  more  uncertainty  about  earnings  in  some  industries  than  in 
others,  and  the  differences  persist  over  the  sample  time  period.   Differences 
in  predictability  across  years  can  arise  if  there  is  more  aggregate  uncertainty 
in  some  years  than  in  others.   Recall  that  the  dependent  variable,  Ej^-;*.,  is 
purged  of  annual  average  forecast  errors.   The  y,  rneasure  year-by-year  average 
dispersion  around  the  annual  means. 

The  form  of  equation  (3)  is  convenient  for  statistical  testing.   Tests  for 
homogeneity  in  accuracy  are  linear  tests  on  groups  of  parameters.   The  test  for 
homogeneity  in  forecast  accuracy  among  analysts,  for  example,  is  an  F  test  that 
the  y,  all  are  equa] .   The  results  of  homogeneity  tests  in  equation  (3)  are 
reported  in  Table  5.   The  tests  reject  homogeneity  among  industries  and  across 
years,  but  fail  to  reject  homogeneity  among  analysts.   That  is,  there  is 
evidence  in  this  sample  that  forecast  accuracy  differs  by  year  and  by  industry, 
but  there  is  no  evidence  that  forecast  accuracy  differs  across  the  191  analysts 
in  the  sample. 


5.   Summary  and  Conclusions 


The  model  was  estimated  using  firm  effects  in  place  of  industry  effects. 
The  results  are  very  similar  to  those  reported  here.   Industry  effects  are 
slightly  more  significant  than  firm  effects. 
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In  this  paper,  I  have  considered  the  question  of  individual  forecasting 
ability  by  examining  forecasts  of  earnings  per  share  made  by  191  analysts  in 
the  period  1975  through  1981.   There  is  no  evidence  in  this  sample  that 
analysts  differ  in  forecast  accuracy  to  a  statistically  distinguishable  extent. 
Forecast  accuracy  is  defined  here  as  mean  squared  error  on  the  portfolio  of 
firms  for  which  the  analyst  produces  forecasts,  through  time,  controlling  for 
differences  in  predictability  among  industries  and  across  years. 

There  is  evidence  in  the  sample  that  average  annual  forecast  errors  vary 
significantly  from  year  to  year.   This  is  expected,  especially  over  longer 
forecasting  horizons,  due  to  unanticipated  macroeconomic  events  which  affect 
many  firms  in  a  similar  manner,  and  which  differ  from  year  to  year.   Analysts' 
forecast  accuracies  are  therefore  measured  in  terms  of  deviations  around  the 
annual  average  of  all  forecast  errors. 

Some  caveats  are  worth  noting  about  the  results.   While  the  number  of 
analysts  included  in  the  sample  is  moderately  large,  the  necessity  of 
sufficient  data  for  reliable  estimates  excluded  the  vast  majority  of  the 
analysts  in  the  database  from  the  sample.   To  avoid  attributing  forecasts  made 
by  two  different  individuals  to  a  single  one,  a  conservative  approach  to 
identifying  analysts  is  employed.   This  approach  almost  surely  results  in  the 
exclusion  of  analysts  who  change  jobs  frequently  from  the  sample.   If  analysts 
who  change  jobs  frequently,  but  who  remain  in  the  database  employed  by 
different  financial  service  institutions,  do  so  because  they  are  superior 
analysts  who  are  bid  away  by  competing  firms,  then  the  identification  and 
sample  selection  processes  may  exclude  these  superior  analysts. 

I  have  examined  only  one  component  of  analysts'  services  to  their  clients, 
ability  to  forecast  earnings.   Forecasting  ability  is  worthy  of  study,  because 
earnings  forecasts  are  useful  to  academics  as  well  as  to  investors.   From  a 
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practical  standpoint ,  earnings  forecasts  possess  advantages  of  relative 
objectivity  and  quantif iability ,  and  of  data  availability.   It  may  be  the  case, 
however,  that  analysts  are  valued  more  for  their  other  services  to  clients, 
such  as  buy/sell  recommendations,  than  for  their  earnings  forecasting  ability. 
If  it  is  the  case  that  analysts  are  not  compensated  for  producing  highly 
accurate  forecasts,  then  it  is  not  surprising  that  analysts  are 
indistinguishable  in  terms  of  forecast  accuracy. 


TABLE  1 


The  Effects  of  Sample  Selection  Criteria 


Selection  criteria ; 


tt  Firms   #  Forecasts    #  Analysts 


(1)  December  31  yearend, 

with  at  least  1  forecast 
in  I/B/E/S  in  each  year, 
1975-1981 


508 


n.a. 


n.a. 


(2)   Annual  EPS  available 
on  1982  Compustat 


497 


n.a . 


n.a, 


(3)   Analyst  forecast(s) 

available  in  I/B/E/S  120 
trading  days  before  yearend 


457 


38611 


3041 


(4)   Analysts  with  at  least 
30  forecasts 


445 


16970 


323 


(5)   Analysts  in  (4)  with 

forecasts  in  at  least  4  years 


433 


11787 


208 


(6)  Analysts  with  at  least  30 

forecasts  in  at  least  4  years, 
no  forecast  more  than  200 
trading  days  old 


428 


10586 


IPl 


TABLE  2 


Given  other  selection  criteria*,  how  does  selection 
on  forecast  age  alter  the  sample? 


Forecasts 

Analysts 

Firms 

Criterion 

# 

%  sample 
reduced 

0.0% 

208 

%  sample 
reduced 

0.0% 

433 

%  sample 
reduced 

No  age  selection 

11787 

0.0% 

Age  <  200  days** 

10586 

10.2 

191 

8.2 

428 

1.2 

Age  <  100  days 

7343 

37.7 

142 

31.7 

411 

5.1 

Age  <  50  days 

2786 

76.4 

56 

73.1 

336 

22.4 

Age  <  25  days 

677 

94.3 

11 

94.7 

258 

40.4 

Forecasts  made  since 
2nd  quarter  EPS 
announcement  268     97.7         4   98.1        162    62.6 


*  Other  selection  criteria  are:  analysts  must  have  at  least  30  forecasts 
in  at  least  4  years  in  the  I/B/E/S  database,  for  firms  with  December  31 
yearends  with  earnings  data  on  COMPUSTAT. 

**  Forecast  ages  are  measured  in  trading  days,  as  defined  on  the  CRSP 
trading  day  calendar. 


TABLE  3 
Industry  Representation  in  Sample 


2-digi 
SIC  CO 

t 
ide 

Industry  name 
Paper 

Fore 

650 

casts 

17 

Firms 
% 

26 

6.1% 

4.0% 

28 

Chemicals 

1533 

14.5 

41 

9.6 

29 

Petroleum  refining 

1150 

10.9 

20 

4.7 

33 

Primary  metals 

350 

3.3 

10 

2.3 

35 

Machinery 

507 

4.8 

29 

6.8 

36 

Electrical  & 
electronic 

306 

2.9 

23 

5.4 

49 

Elec. ,  gas  & 
sanitary  svcs. 

1892 

17.9 

63 

14.7 

60 

Banking 

1073 

10.1 

33 

7.7 

63 

Insurance 

314 

3.0 

9 

2.1 

7775     73.4%         245    57.2^ 


Other        40  2-digit 

industries  2811    26.6  183   42.8 


total  10586    100.0%         428   100.0^ 


TABLE  4 
Average  Annual  Forecast  Errors* 


#  of 

Mean 

Standard  e 

Year 

Observations 

forecast  error 

of  mean 

1975 

694 

$ 

0.00 

$   0.05 

1976 

1118 

-0.21 

0.04 

1977 

1400 

-0.25 

0.03 

1978 

2048 

0.08 

0.03 

1979 

1931 

0.12 

0.03 

1980 

1702 

-0.09 

0.03 

1981 

1685 

-0.37 

0.03 

F{6,  10571)   ^   35.99** 


*  The  average  forecast  errors,  measured  in  dollars  per  share,  are  computed  via 
least  squares  by  pooling  across  firms  and  analysts  within  years. 

**  The  F-statistic  tests  rejects  the  hypothesis  of  homogeneity  in  annual 
averages  at  conventiona]  significance  levels. 


TABLE  5 

Tests  for  Homogeneity  in  Forecast  Accuracy* 

Source  of 
Variation 

Year  F(6,  10333)         =    5.64** 

Industry  F(48,  10333)        =    5.97** 

Analyst  F(190,  10333)       =    0.94*** 


*  The  homogeneity  tests  are  based  on  a  fixed  effects  model,  with  forecast 
accuracy  defined  as  average  squared  deviations  from  the  annual  mean 
forecast  error. 

**  Homogeneity  rejected  at  conventional  significance  levels. 

***  Homogeneity  not  rejected. 
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